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Abstract 

We present a theory of quantum stabilizer turbo-encoders with unbounded minimum 
distance. This theory is presented under a framework common to both classical and 
quantum turbo-encoding theory. The main conditions to have an unbounded minimum 
distance are that the inner seed encoder has to be recursive, and either systematic or with 
a totally recursive truncated decoder. This last condition has been introduced in order 
to obtain a theory viable in the quantum stabilizer case, since it was known that in this 
case the inner seed encoder could not be recursive and systematic in the same time. 

1 Introduction 

Turbo-codes are a class of codes with very good properties. Provided that the outer encoder 
and the inner encoder are chosen according to certain requirements, namely, that the minimum 
distance d c of the outer encoder is greater than or equal to 3, and the inner encoder is 
recursive and catastrophic, then the random turbo-encoders built by linking these encoders 
by the means of an intermediate random interleaver have a minimum distance which behaves 

dg-2 

asymptotically as N d ^ where N is the size of the input; if the minimum distance of the 
outer encoder is equal to 2, then the minimum distance attains log N at best for a particular 
choice of the interleaver, whereas the minimum distance is bounded for a randomly chosen 
interleaver. On the other hand, if the inner encoder is moreover non-catastrophic, these 
turbo-codes show good decoding performances for the symmetric channel as well as for the 
erasure channel, under a regime near to the Shannon limit, and with an iterative decoding 
algorithm with linear complexity in N. The property of having an unbounded minimum 
distance is lost if the inner encoder is either non recursive or non systematic, and the linear 
iterative decoding algorithm becomes useless if the inner encoder is catastrophic, since the 
decoder can not extract any useful information about the local probability distribution on 
the state of each bit. 

1.1 The classical argument and where it fails in the quantum case 

Attempts [2] were made to design quantum stabilizer turbo-codes with unbounded minimum 
distance and good decoding performance. However, it was found in [2] that quantum inner 
convolutional encoders can not be recursive and non catastrophic in the same time, nor can 
they be, as a consequence, recursive and systematic. In this paper, we overcome the difficulty 
and introduce a different set of conditions under which quantum stabilizer turbo-encoders 
have an unbounded minimum distance. Our result is partially inspired by pp. The counting 
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argument we adapt in this paper comes by [lj and has the following outline. First, upper 
bound the number a ou t(w) of possible inputs of the outer encoder for which the output has 
weight w, and the number ai n (w, < d) of possible inputs of the inner encoder for which the 
output has weight < d. Using the randomness of the interleaver, this implies an upper bound 
on the number of possible inputs of the turbo-encoder such that the weight of the intermediate 
output (after applying the outer encoder) is w and the weight of the final output is < d. By 
summing this upper bound over all possible values of w one gets an upper bound on the 
number of possible inputs for which the output weight is less than d. Kahale and Urbanke 

d c -2 

show that if d is 0(N d c ), such a sum tends to as N tends to infinity. A very crucial 
hypothesis in the counting argument which can not be used in the quantum case is that 
one can suppose that w < d because the inner encoder is systematic. This is where a new 
hypothesis has to be made in the quantum case, in order to have an acceptable upper bound 
for the sum in the new regime where w > d. A possible new hypothesis we investigate in this 
paper is that the truncated decoder, ie the decoder where the ancillary positions are thrown, 
corresponding to the inner encoder is totally recursive. 

1.2 Results obtained under the common framework 

In order to highlight the similarities and the differences between the classical and the quantum 
stabilizer settings, we define a formal framework to which both settings correspond as a 
particular case. The main idea behind this framework is to consider encoders as operations 
which act on a group of errors V (F2 or the factor Pauli group depending on the settings) rather 
on the real input states. Under this formalism, a turbo-encoder is the concatenation of three 
operations: an outer encoder CfJ^ out equal to the blockwise repetition of a seed encoder C ou t, 
a random interleaver, and an inner convolutional encoder Ci n ^ in . We generalize the concept 
of recursive and systematic encoders, and introduce another concept of total recursiveness. 
We also write the definition of the distance d c of an encoder, and introduce the degenerate 
distance d q . This distance is not different from d c in the classical setting, but it has a very 
important role in the quantum case since d q comes instead of d c in the expression of the lower 
bound on the minimum distance of the turbo-encoder. Some concepts like the detours of a 
convolutional encoder are more delicate and need to be redefined to comply with the quantum 
case. We then show the following results. The first result is only useful in the quantum setting 
since this is where \V\ > 2. The second result is only useful in the classical setting, since in 
the quantum setting the conditions on the inner encoder are impossible to satisfy (see [2]). 
Actually, the second result corresponds to the classical result stated by Kahale and Urbanke 
in [1] and the proof hereby follows the unpublished proof of Kahale and Urbanke. 

Theorem 1.1. Consider a turbo-encoder T/v of size N , where the interleaver is randomly 
chosen according to a uniform distribution. Let d c and d q be respectively the distance and the 
degenerate distance of the outer seed encoder. If all the following conditions are realized: 

• \V\>2 

• d q > 2 ( case 1) or d c > d q = 2 ( case 2 ) 

• the inner seed encoder is recursive, and the associated truncated decoder is totally re- 
cursive 

then with probability going to 1 as N — > 00, the distance of the turbo-encoder is greater than: 
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• case 1; N a for all a < dq d 2 

• case 2: a ^°f Q ^ N for all a < d c - 2 

Theorem 1.2. // a// the following conditions are realized: 

• d q > 2 

• the inner seed encoder is recursive and systematic 

then with probability going to 1 as N — > oo, the distance of the turbo-encoder is greater than 
N a for all a < ^ 

In the counting argument needed to prove these results, we will separate the sum into 
three partial sums. The first partial sum corresponds to the case where w < d as in the 
classical case. The second and the third partial sums correspond to the regime where w > d, 
and are separated into the case where w is sublinear and the case where w is linear with 
respect to the input size. These three partial sums will be upper bounded by an expression 
tending to when the input size tends to infinity, under a set of hypothesis proper to each of 
them. The article is organized as follows. In Section 2, we present the common framework. In 
Section 3, we focus on the definition and some properties of the inner encoder: we introduce 
the formalism of convolutional morphisms, define the notions of recursiveness, systematicity 
and total recursiveness, and define the speed r/ of a convolutional morphism. In Section 4, we 
establish two upper bounds on the weight distribution aj n (u>,< d) (and actually ai n {w,d)) 
of the inner convolutional encoder. The proof is delayed to Appendix A. In Section 5, we 
establish two upper bounds on the weight distribution a ou t(w) of the outer encoder. In Section 
6 we state the upper bounds on the three partial sums needed to proof each of Theorem 11.11 
and Theorem 11.21 and we assemble these bounds to prove Theorems 11.11 and 11.21 The proof 
of the partial sums is delayed to Appendix B and relies on the bounds established in Sections 
4 and 5. 

2 The common framework 

Focusing on errors propagation under an encoding process, rather than on the code or the 
encoding process itself, enables to put linear classical codes and quantum stabilizer codes 
under one common framework, and to grasp distance properties of the underlying code. For an 
introduction to the theory of stabilizer codes, the reader can refer to [ref codes stabilisateurs] . 
Errors to consider in order to know the minimum distance of a code are bit flip errors for 
linear classical codes and Pauli errors for quantum stabilizer codes. In the first case this is 
the direct consequence of the definition of the minimum distance of a classical code; in the 
second case, this comes from the fact that a subspace is a quantum error correcting code for 
a set of errors if and only if it is a quantum error correcting code for a vector space basis of 
these errors. We will call classical and quantum setting, respectively, the model of encoding 
and error propagation for linear classical codes and for quantum stabilizer codes. 

2.1 Encoding protocol 

In the classical setting, a (n, k) code is a linear subset of 2 k elements of the set of n bit states 
Fg. Such a code is the image of an encoding protocol from F2 to F?j, which first appends 
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n — k ancillary bits set to and then applies a F2-group automorphism V of It is thus 
the set of all V(t/j,O n ~ k ) where ip describes the set of all fc-bits states. 

In the quantum setting, a (n, k) code is a 2 fe -dimensional C-subspace of the space of n- 
qubits states H® n , where T~L = {a\0) + (3\1), (a, (3) € C 2 } designates the one qubit space. It 
is the image of an encoding protocol from U® k to %® n , consisting in the addition of n — k 
ancillary qubits set to |0) followed by a C- vector space automorphism V of l~L® n . Again it 
is the set of all V(ip <g> |0) n ~ fc ) where describes the set of all fc-qubits states. V is more 
precisely a Clifford transformation, so that it stabilizes the group of Pauli transformations by 
conjugation. 

Putting the encoding in the form of this protocol has the advantage to identify n — k 
positions which will carry all the information about the error given by the decoding process 
V" 1 . 

2.2 Error propagation: the encoder 

Suppose that in the classical setting, an n-bits state (ip,O n ~ k ) is subject to an error E = 
E\...E n € Fg, where Ei = 1 if and only if a bit flip happens at position i. After applying V, 
we get the state V(E + ip) = V(E) + V(ip), which means that the effect of V on the set of 
errors F2 is the group automorphism V itself. 

In the quantum setting, suppose that an n-qubits state (ip ® |0) n_fc ) is subject to a Pauli 
error E € G n . After applying V, we get the state VEip = (yEV~ l )Vil), which means that 
the effect of V is to map each element E of G n into VEV -1 , which also belongs to G n since 
V is a Clifford transformation. The operation E — > VEV 1 is a group automorphism of G n . 
Elements of G n contain a global phase in —1, — i} which can be omitted for E as well 
as for VEV~ 1 without any loss of information; this corresponds to considering the quotient 
Pauli group G n /Z(G n ), equal to the n-fold cartesian product of the quotient Pauli group of 
4 elements G\jZ{G\) = {I,X,Y,Z}. The induced action of the operation E — s> VEV^ 1 on 
the quotient Pauli group is, again, a group automorphism. 

Thus it is possible to unify both settings by writing that F2 and {/, X, Y, Z} constitute a 
group of errors V with neutral element /. The group of errors also needs to verify a property 
with respect to the decoding step and stated in the upcoming definition of the common 
framework. We then say that the automorphism of V n engendered by the encoding process, 
together with the knowledge of k to separate the information carriers positions from the 
ancillary positions, constitute an [n, A;] encoder C. Let us now push the comparison between 
the two settings a little further, by underlining the concepts of undetected errors and harmless 
errors. 

2.3 Types of errors and the distances d c and d q 

Suppose now that a non trivial error affects the state obtained after the encoding operation. 
Since the [n, k\ encoder C is an automorphism, we can write this error in the form C(E) 
with E a non trivial error. When the decoding operation V -1 is performed, the state left is 
E(ifi, n ~ k ) or E(ip £3 |0)™~ fc ) depending on the setting. In both settings, one sees that if the 
last n — k positions are left unchanged by E, the error C{E) is undetected. This happens in 
the classical setting if the last n — k coordinates of E are equal to 0, and in the quantum 
setting if the last n — k coordinates of E are in the set {/, Z} of errors acting trivially on the 
qubit |0). In the case of an undetected error, we are left with a state in the form (i{)',0 n ~ k ) 
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or (tp' <g> |0) n_fc ) depending on the setting; if tp' = ip, the error C(E) is harmless, otherwise it 
is harmful. In both settings , the minimum distance of the code is the minimum Hamming 
weight of a harmful error. We simply call this value distance and write d c , whereas the 
minimum Hamming weight d q < d c of an undetected error will be called degenerate distance. 

2.4 Definition of the framework 

Definition 2.1. • A group of errors V is a finite group with composition law *, neutral 
element I, which contains a strict subgroup Z called the group of undetected syndromes. 
An element of V is called a letter. An element of V n where n G N* is called an error. 
The weight of an error E G V n is #{i G [1, nj,Ei / /}, where E{ is the i-th coordinate 
ofE. 

• An [n, fe] encoder, where n> k, is an isomorphism from V k x p n ~ k to V n . 

• Let E G V n , E / I n , and C an \n, kj encoder. For 1 < i < n, let C- 1 (E) i be the i-th 
coordinate of C~ l {E). Then E is undetected for C if: 

ViG lk + l,n},C-\E) t eZ 

E is harmless for C if it is undetected for C and: 

ViG ll,kj,C- 1 (E) i = I 

E is harmful for C if it is undetected for C and: 

3i£ {l,kj :C-\E)i^I 

• The distance d c of C is the minimum weight of a harmful error for C 

• The degenerate distance d q of C is the minimum weight of an undetected error for C 

Let us also introduce a few writing conventions. As presented, an error E is a sequence of 

elements of V . The i-th element of this sequence is written Ei and we write E = E1.E2 En 

where iV is the size of E. E will also often be seen as a concatenation of errors of smaller size, 
each playing a particular role with respect to the encoding protocol. We have already seen the 
standard encoding protocol, in which the first k positions of the input carry the information, 
the last n — k positions are an ancilla, and the n positions of the output carry the encoded 
information. Restrictions of an error to these respective positions are called information, 
stabilizer, and physical errors, and written L, S and P, such that in a standard encoding pro- 
tocol, the error affecting the input can be written E = (L, S), and the output can be written 
C(E) = P. As will be presented with the convolutional encoder, some positions play the role 
of a memory, in which case the restriction of an error to these positions is written M. The 
weight of the information, ancilla, physical and memory parts of E are written respectively 
\E\l, \E\s, \E\p and \E\m- Moreover, the number of elements of E in the ancilla part which 
belong to V\Z is called the weight of the detected syndromes and written \E\x- 
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3 The inner encoder: definitions and properties 



3.1 Convolutional encoders, truncated convolutional decoders 

Let us now define convolutional encoders and truncated convolutional decoders. Instead of 
considering simply the inverse transformation of a convolutional encoder, we will remove 
the stabilizer part of the output, and this is why we use the word truncated. Likewise, a 
similarity will be drawn between these two transformations, from which we can generalize 
simply what happens in a recursive encoder to what happens in a totally recursive decoder. 
A convolutional encoder is built by using a seed [n, k, m] encoder C. The seed transformation 
related to a truncated convolutional decoder is a truncated decoder: 

Definition 3.1. An [n, k,m\ encoder of memory size vn, information size k, and stabilizer 
size n — k, is an isomorphism from V m x V k x p n ~ k to V n x V m . The truncated decoder of an 
[n, k, m] encoder C , is the application C from V" 1 x V" to V k X V m obtained, by truncating the 
output ofC^ 1 to its information and memory parts, more precisely, for all (M', P) G V m xV n , 



One very important thing to notice here is that an encoder and a truncated decoder are 
both particular cases of a morphism from V m x V k x V s to V n x V m . For an encoder, we 
have n > k and s = n — k, whereas for a truncated decoder, we have s = and the roles of k 
and n are switched. Let us thus say that a morphism from V m x V k x V s to V n x V m is an 
[n, k, s, m] morphism. In this case, an [n, k, m] encoder is an [n, k,n — k, m\ morphism, and 
the truncated decoder associated to an [n, k, m] encoder is an {k,n, 0,m] morphism. 

A convolutional encoder Cat of size N and parameters (m,k,n) is an isomorphism from 
<pm x ^pk x <pn-k^ to (p n ^ N x V m ■ An input error is the concatenation of a memory error 
with alternations of one information and one stabilizer error, and an output error is the 
concatenation of N physical errors ended by a memory error. The convolutional encoding is 
done by repeating N times a given [n, k, m] encoder C, which acts each time on one memory 
error, one information error and one stabilizer error, and produces one physical error and 
one memory error. Thus at the i-th step, the intermediate error is the concatenation of i — 1 
physical errors, one memory error, and N — i couples of information and stabilizer errors. Thus 
formally, we define a convolutional encoder by introducing N intermediate transformations 
acting on a total of iV + 1 intermediate groups. A truncated convolutional decoder Cn 
is a morphism from V m x (p n }® N to (J> k )® N x V m ', obtained by applying successively a 
truncated decoder C. Its input is a memory error concatenated with N physical errors, and 
its output is the concatenation of N information errors ended by a memory error. Let us 
define a convolutional encoder and a truncated convolutional decoder, after defining, for each 
of them, the N + 1 intermediate groups and the N intermediate transformations. 

Definition 3.2. Let C be an [n, k,mj encoder and let JVeN*. For 1 < i < N + I, let: 



if: 



C-\P,M') 



(M,L,S) 



then: 



C(M',P) 



(L,M) 



Pi,N = (V n ) 



in\®i— 1 




fc.®JV-i+l 
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and let: 



For 1 < i < N, let C l N be the isomorphism from Pi t N to Pj+i^ defined by: 

C\j(Pi, Pi-i, Mj_i, Li, Si, Ljv, Sn) = (Pi, ■ Pi, Mi, Li + \,Si + i, Ln, Sn) 

where 

(P i ,M i ) = C(M i - 1 ,L i ,S i ) 
Let also C l N be the morphism from V-i-i^n to V-^n defined by: 

C 1 n (Ln, Lj+i, Mi, Pi, Pi) = (Ljv, Li, Mj_i, Pj_i, Pi) 

where 

(L i ,M i _ 1 ) = C(M i ,P i ) 
The convolutional encoder Cn is the isomorphism from Vi t N to Vn+i,n defined by: 

Cn = o 1 o ... o Clf 
The truncated convolutional decoder CV is the morphism from V~n~i,n to P-i t N defined 



by: 



Cn = C N o ... o C^j 1 o 0$ 



Notice that if one runs the truncated convolutional decoder Cn on the output of the 
convolutional encoder Cn , all the intermediate errors Mj during the truncated convolutional 
decoding stage correspond to the Mj of the convolutional encoding stage, and the information 
errors obtained at the end of the truncated decoding are equal to the initial information errors 
before the encoding. The memory errors Mj for 1 < i < N will play an important role later; 
let us call Mj the ith intermediate memory error. Also, let us call ith intermediate sequence 
the sequence (Pi, Pj, Mj, Lj+i, Sj+i, ...,Ln, Sn) obtained at the step i of the convolutional 
encoding. A convolutional encoder and a truncated convolutional decoder are both a case of 
a convolutional morphism defined as follows. 

Definition 3.3. Let C be an [n, k, s, m] morphism and let N G N*. For 1 < i < N + 1, let: 

V^n = (V 71 )® 1 ' 1 x V m x [V k x V sf N - i+1 
For 1 < i < N, let C l N be the isomorphism from Pi,N to Pj+i,^ defined by: 

Cn(Pii Pi-i, Mi_i, Li, Si, Ln, Sn) = (Pi, Pi, Mi, Lj+i, Ln, Sn) 

where 

(Pj,Mj)=C(Mj_i,Lj,Sj) 
The convolutional morphism Cn is the isomorphism from Vi t N to Vn+i,n defined by: 

Cn = Cn ° C^ 1 o ... o C N 

It is useful to separate the physical part and the memory part of an error output by a 
convolutional morphism: 
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Definition 3.4. Let N € N*. tin and are the maps from Vi : n to respectively (V n ) 
and V' m , such that Cn = {^n-,^n)- 

We also define a convolutional encoder and truncated decoder for inputs of infinite 
size. They are both a particular case of the following convolutional morphism. 

Definition 3.5. The convolutional morphism is the morphism from V m x (V k x - p s )® N 
to (-p n )® N ; such that for any infinite input sequence E = (M, Li, Si, L2, S2, ■■■), and for all 
i 6 N*, the sequence of the first i errors of Coo(E) is equal to 7Tj(M, L\, Si, Lj, Si). 

The following lemma is a direct consequence of the convolutional construction: 

Lemma 3.6. For all 1 < i < N, and for all input sequences (M, L\, Si, Ljy, Sn): 

Cn{M,Li,Si, ...,L n , Sn) =iTi(M,Li,Si, Li, Si). 

Cn-i{ii{M, Li, Si, Li, Si), Sj+i, L^, Sn) 

3.2 Recursive, systematic encoders, and totally recursive decoders 

We define here the notion of a recursive encoder and a systematic encoder, which are an 
extension of the already existing notions known for classical convolutional encoders. We also 
present the new notion of a total recursive decoder. The word total is used to emphasize the 
fact that for a decoder to be totally recursive, no condition is required about the stabilizer 
weight of the output sequence of the decoder. 

Definition 3.7. An \n,k,s,m\ morphism C is recursive if: 

VE G V m x [V k x V s ) , \E\ M = 0, \E\ L = 1 and \E\ X = {C^E^ = 00 

An [n, k,m\ encoder C is recursive, or a truncated decoder C is totally recursive, if the 
\n,k, s,m] morphism they correspond to is recursive. Ln other words, an \n,k,m\ encoder C 
is recursive if: 

\/E G V m x (v k x <p n - k y l \ \E\ M = 0, \E\ L = 1 and \E\ X = => \Coo(E)\ = 00 

An \n, k, m\ truncated decoder C is totally recursive if: 

VEeTx (pn-^N ^ m = and = 1 ^ \Coo(E)\ = OO 

An \n, k, m] encoder C is systematic if: 

ViV GN,VBG TV, ICat^)! > 
An equivalent characterization of recursiveness will come after defining the sets Mo and 

Mi. 
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3.3 Memory errors, speed, and characterisation of recursive encoders 

In this subsection, let C be an [n, k, s, m] morphism. Memory errors have two kinds of 
behaviour. Consider an infinite input sequence E composed by concatenating a memory 
error M and infinitely many couples of one information error and one stabilizer error, where 
all the information errors are equal to /, and all the stabilizer errors belong to Z s . Then 
depending on M, the corresponding output will either have an infinite weight independently 
of the sequence of stabilizer errors, or there will exist a stabilizer sequence for which the 
output weight is finite. This is what we intend to describe in the definition of these two sets: 

Definition 3.8. 

M = {M eV m /3(S 1 ,S 2 ,...) eZ sm /\C 00 (M,I,S 1 ,I,S 2 ,...)\ < 00} 
Mi = {M eV m MS 1 ,S 2 ,...) eZ sm ,\C 00 (M,I,S 1 ,I,S 2 ,...)\ = 00} 

Let us also define a set useful for the characterization of a recursive morphism, the set 
of memory errors I, accessible by starting with the I memory error and applying C a finite 
number of times on an input of information weight and detected syndrome weight 0: 

Definition 3.9. I = { fi N (I, I, Si, IS N ), N G N,(S U ...,S N ) G {Z S ) N } 

The goal of the next two lemmas is to prove that if a memory error belongs to Mi, then 
the weight of a finite output Cn(I,I, Si, ...,/Sjv) will be proportional to its length N. The 
constant of proportionality is in the form I/77 where the integer 77 depends on C. 

Lemma 3.10. There exists a positive integer r\ such that: 

VMS Mi, V(Si,...,S„) G (Z s fM^(M,i-,Si,...,7,S^)| > 1 
The smallest such r\ is called the speed of the morphism C. 
Proof. We prove the lemma by contradiction by supposing that: 

Vr? G N*, 3M„ G Mi, 3(5?, S£) G (Z s fV \tt v (M, I, S?, S%)\ = 

Since the sequence (M v ) v€ ^ has its values in the finite set Mi, there exists M G Mi such 
that \{r) G N*/ M v = M}\ = 00. 

Now let us define a sequence of stabilizer errors (Sj)j e N* G (Z S ) N such that: 
V2 G N*, \{q>i: (5?, Sj) = (Si, 5)}| = 00 

• The sequence (5?)^ s n has its values in the finite set Z s . Thus, define Si as an element 
of Z s such that: 

\{rj > 1 : S? = Si}| = 00 

• Suppose that for a given i G N*, we have defined i stabilizer errors Si,..., Si, such that 
for all j <i: 

\{ V >j:(Sl...,S]) = (S 1 ,...,S j )}\ = oo 
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Let Nj = {q > i + 1 : (SJ 7 , S^) = (Si, Si)}, which by hypothesis is infinite. Since 
the sequence (S 1 j l +1 ) rje n i has its values in the finite set Z s , we can define Sj + i as an 
element of Z s such that: 

\{V G Nj : S? +1 = S l+1 }\ = oo 

which implies: 

\{V > i + 1 : (Si S? +1 ) = (S u S i+1 )}\ = oo 

Now let us use this sequence (S'i)igN* € (2 S ) N to produce a contradiction. Let z G N*, 
and let 77 > i such that (S?, S?) = (Si, S;). Then: 

^(M,/,^, ..,#)! = \t H (M,I,SI...,SV)\=0 

Since this is true for all i e N*, \C 00 (M,I,Si,I,S2,—)\ = 0, which contradicts the fact 
that M G Mi. □ 

Lemma 3.11. For all N > 0, and for all (Si, ...,Sn) G (Z s )® N the function: 

^r>m y jy m 

M ^ m (M,I,Si,...,I,S N ) 

stabilizes the set Mi . 

Proof. Let7V>0and(5i,..., 1 S iV ) G (Z s ) 0Ar . Let M G Mi, and let (P, M') = C N (M, I, Si, I, 
For all (S N+ i,S N+2 ,...) G (2 S )® N : 

Coo(M, I, Si, I, S2, •••) = (P, Coo(M', I, Sn + i,I, Sn+2, •••)) 

And since M G Mi, {C^W, I, S N+ i, I, S N+2 , ...)| = 00. 

Thus M' e Mi. □ 

As a consequence: 
Lemma 3.12. For all N > 0, M G Mi, and (Si, ...,Sjv) G (Z) 8 ^: 

|7rAr(M,/,Si,...,I,S^)| > L^J 
Proof. Let M G Mi . We prove the lemma by recursion on p = [N/ 77 J . 

For p = 0, the property is clearly true. 

Now suppose the property is true for a given p — 1, where p > 1. 

Let iV > such that [A^/r/J = p, and let (Si,..., S^) G (Z) s0Ar . The following relation 
holds: 

ttn(M, I, Si,..., I, Sn) = tt^(M, I, Si, S v ). itN-i)(lh)(M, I, Si, I, S V ),I, S v+1 , Sat) 
Since M G Mi, /J, V (M, I, S\, I, S v ) G Mi, which implies by the recursion hypothesis: 

\ir N - ri (n ri (M,I,Si,...,I,S ri ),I,S ri+ i,...,I,S N )\ >p-l 
And since \Tr v (M, I, Si, /, S v )\ > 1, the property is also true for p. □ 
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Another consequence is the following characterization of recursive morphisms: 

Lemma 3.13. C is recursive if and only if for all (M,L,S) G I * V k * Z s with \L\ = 1, 
fi(M,L,S) eMi. 

Proof. Let E be an infinite input sequence of memory weight 0, information weight 1, and 
such that all its stabilizer letters are in Z s . Let K be the position of the only information 
error L of weight 1. E is in the form: 

E = (I, I, Si,..., I, Si-i,L, Si, I, Si+i,I, Si + 2, ...) 

Let (P,M) =C i _i(/,/,5i,...,/,5 i _i). Then: 

Coo(E) = P- Coo[M, L, Si, I, Si+i,I, Si + 2, ...) 

= P. 7r(M, L, Coo^M , L, Si) , I , Si + il , Si + 2, ■ ■■) 

Thus, |Coo(-E) | = oo for all £" which verify the assumptions above, if and only if [i(M, L, Si) G 
Mi for all i > 1, L of weight 1, S% in Z s , and M in the form /^j_i(J, /, Si, S^-i) where 
(5i, fii-i) G or in other words for all Mel. 

□ 

4 Weight distribution of the inner encoder: upper bounds in 
the recursive and the totally recursive cases 

In this section, we will establish two upper bounds to the weight distribution of a con- 
volutional encoder. What is meant precisely by the weight distribution is the number of 
possible (memory and information) subsequences (M, L\, Ln) of an input sequence E = 
[M, Li, Si, Ln, Sn), for a given weight w of the input subsequence and a given weight d 
of the output sequence. The first upper bound comes provided that the seed encoder C is 
recursive, and the second one comes provided that the seed truncated decoder C is totally 
recursive. We will establish an upper bound for a recursive morphism, and derive the two 
desired results as a corollary. 

4.1 Trace and detours 

Before going to the point of the upper bounds, we first need to exhibit a characteristic 
behaviour of recursive morphisms; mainly, we will first show that the trace of an input se- 
quence of a convolutional morphism is a concatenation of detours. By introducing the con- 
cept of trace, this phenomenon of detours is a generalisation of the phenomenon described 
by [1] in the case of a classical convolutional encoder based on a recursive seed encoder. 
Consider a recursive [n, k, s, m] morphism C. Let JV be a positive integer, and consider 
E = (M, Li, Si, -£jVj Sn) = Ei ... Ej\f an input sequence to the convolutional morphism 
Cn, where J\f = m + N(k + s). Thus we have M € V m , and for all % £ [1, n], Lj G V k and 
Si G Z s . Let (Pi, ...,Pn,M') = Cn{E) be the output sequence. 

Let wm = \E\m = \M\ and wl = \E\l = \{Li, ...,L^)\. By looking at the information 
part (Li, Ljv) as a sequence of kN letters, label by 1 < pi < ... < p WL < kN the positions 
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of the wl non identity letters, and for each i € [1; wjj\, let Ni = [pj/fe] so that the ith non 
identity letter is part of the error L^ i . 

Define the ith truncature of E as the sequence: 

E \i = {M,Li,Si, L Ni _ 1} Sn^, L' N _, SnJ 

where (L±, _ 1 , L' N ) is obtained by replacing all the letters in (L±, -LjvJ after the 
position pi by an /. Let us also set po = 0, iVo = and E\ = M. Define Mj as the memory 
error of the output when running on E\f 

Mi = n N .(E\i) 

Set also Mq = M. This enables to define the trace of E: 

Definition 4.1. The tract of E is the sequence (bo, b WL ) of elements of {0,1} such that 
for all ie lO;w L j, M;€M 6 .. 

The following result holds as a direct consequence of this definition: 

Lemma 4.2. bj = 1 if and only if for all sequences (Sk)ken* of stabilizer errors in Z s : 

ICoo^. (/,5i,/,S 2 ,...))| =oo 

Proof, bi = 1 if and only if Mj € Mi, thus if and only if for all sequences (Sk)k£N* of 
stabilizer errors in iJ s : 

ICooCMi,/,^,/,^,...)! = oo 

The relations: 

Coo(£ v . (/,5i,/,5 2 ,...)) =7T JVi (£\ i )- C 00 (M i ,/,Si,/,5 2 ,...) 
if i > 1, and if z = 0: 

Coo(E\ . (I,Si,I,S2,—)) = C 00 (Mo,I,Si,I,S2,.-.) 

show that this happens if and only if |Coo(-E\i- <§i, /, §2, ...))| =00. □ 
This implies the following property about the trace of -E 1 : 

Lemma 4.3. For a// i in \0,wl — 11, if h = then bi + \ = 1. 

Proof. Let us suppose that 6j = 0. Then there exists a sequence (S'^)xeN* of stabilizer errors 
in Z s such that: 

|Coo(E v . (I,S' 1 ,I,S 2 ,...))\<oc 
Let (Sat^gn* be a sequence of stabilizer errors in Z s . Then the two sequences 

(^1 S' 2 , •••) 

and 

(I,Si,I,S 2 ,...) 
12 



do not differ in their memory part, differ in their information parts by only the pj+ith letter 
of (Li, Ljv), and differ in their stabilizer parts by a sequence of errors in Z s . Thus one can 
write: 

E v+1 . (I, S U I, S 2 , .-) = E v . (I, S[,I, S' 2 , ...) * AE 
where the sequence AE verifies \AE\m = 0, \AE\l = 1 and \AE\x = 0. Since C is recursive: 

\Coo(AE)\ = oo 

And since is a morphism: 

Coo(£\ i+1 . (/, 5i, J, 5 2 , ...)) = Coo(^ V . (/, S[,I, S' 2 , ...)) * Coo(A£) 
We deduce that: 

|C 00 ( J B\ i+1 . (I,S 1 ,I,S 2 ,...))\ = co 

And since this is true for all sequences (Sk)kgn* of stabilizer errors in Z s , bi + \ = 1. □ 

Definition 4.4. A detour is a finite and non empty sequence ofl, potentially completed with 
a in which case it is called a terminating detour. 

If wl > L according to the previous lemma, either &o ° r b\ is equal to 1. Starting from 
that first 1, the rest of the trace of E is a concatenation of detours, which are all terminating 
except maybe the last detour: 

Lemma 4.5. If wl > 1, there exist c > 1 and c integers v\ < ... < v c < wl such that: 

- v\ = or 1 

- /or a// i in [0; c — 1], (b Vi , is a terminating detour 

- (b Vc , b WL ) is a detour 

As a convention, let v c+ \ = wl + 1, so that (b Vi , b Vi+1 -±) is also the expression of a 
detour if i = c. 

The number c of detours is less or equal to + 1J • Indeed for each i G [1; c — 1], the 
ith detour contains at least a 1 and a 0, whereas the last detour contains at least a 1; this 
implies that wl + 1 > 2(c — 1) + 1, in other words c < + 1. 

4.2 The upper bounds 

We are still with an [n, fc, s, m\ recursive morphism C, and we note n its speed. Let on(w, < d) 
(for dGl) and a N (w,d) (for d G N) be the numbers of sequences (M,Li, ...,L N ) G V m *{V k ) N 
of weight u;, which are part of an undetected input sequence: 

£ = (M, Li, 5i, Lat, Sat) G P m * (P fc * 

such that, respectively, |Cjv(£J)| < d and ^^(E 1 )! = d. The above characterisation of detours 
enables to prove the result (the proof is given in appendix): 

Theorem 4.6. 
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Notice that aj\f(d,w) in the case where C is the truncated decoder C corresponds to the 
number of possible sequences (Pi, Pn,M') of weight d such that the weight of (M, L±, Ln) = 
Cn(Pi, Pn, M') has weight w. This corresponds exactly to the value of a]y(w,d) in the 
case where C is the encoder C. This is what will enable us to obtain the two upper bounds 
on aj^(w,d) concerning the encoder C. By using the following bound on binomials, where 
v < u: 

(:) * 

the two upper bounds can be written in the form of this corollary: 

Corollary 4.7. Let C be an encoder, and let ajy(w,< d) and ajy(w,d) be the number of 
possible sequences (M, L\, Ljv) of weight w, which are part of an input sequence E = 
(M, Li, Si, Ljy, Sjv) where all the Si belong to Z s , and such that Cn(E) has weight respec- 
tively less or equal to d, and equal to d. Then: 
Bound II: If C is recursive: 

w w 

a N (v,<d)<o(ir N ^i:^ 

Bound 21: If C is totally recursive: 

5 The outer encoder 

The outer encoder consists in an [n, kj encoder C repeated blockwisely N times (C and N 
are not the same as the inner encoder). 

Definition 5.1. Let C be a {n,k} encoder and let N G N*. The blockwise encoder C® N is 
an {Nn, Nfcj encoder defined by: 

C® N (Li, List, Si, ...,Sn) = C(Li,Si) C(Ln, Sn) 

where for all i<N,Li£V k and S { £ V n ~ k . 



Lemma 5.2. Bound IE: Let C be an encoder of distance d c and degenerate distance d q > 2. 
For all d > and N > 0, let a® N (d) be the number of input sequences E <E V Nk x Z N( - n ~ k ^ 
such that \E\ L > and \C® N (E)\ = d. Then: 

d — d c | 

a m (d) < 0(l) d (j) dq \id>d c 
= if d < d c 

Proof. Of course if d > N then the bound is true since a® N (d) = 0, so we will consider that 

d < N. Let E = (Li,...,LAr,Si,...,Sjv) where \E\ L > 0, (Si,...,S N ) € (Z n ~ k ) N , and such 
that \C® N (E)\ = d. Let j > be the number of couples (Lj, Si) different from (J, /). In other 
words all these j couples are undetected errors for C, thus we have |C(Lj,Sj)| > d q . Also, 
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since \E\l > 0, at least one of these couples is such that L, t / /. This couple is a harmful 
error for C which implies \C(Li,Si)\ > d c . This first shows that necessarily d > d c , and: 

d>{j- l)d q + d c 

and thus: 

, d — cL . 

J + 1 

For each possible value for j, there are (^) ways to chose the positions of the non zero se- 
quences (Li, Si). Each of these sequences can take less than \V\ n values whereas the remaining 
sequences are all fixed to (1,1). This leads to the bound: 

j= i v J / 

Since [^r^J + 1 < f < eacn t erm °f t ne sum can be majored by the term where j = 
L^J + 1. Thus: 

««M<(L^J + DM" ,l ^ J+1, ( L ^ + 1 ) 



Finally the lemma is proved using the bound on binomials, where v < u: 



u\ /u.e\ v 
< 



V \ V 



□ 

Lemma 5.3. Bound 2E: Suppose that \V\ > 2. Let C be an encoder of degenerate distance 
d q > 2. There exists a constant c e]0, 1[ such that for all d and N: 



a® N (d) < c d (\V\ - l) d 



Nn 
d 



Proof. Let X G V\Z (a non empty set since \V\ > 2), and let Y = X * Z' 1 . For all i G [0, nj, 
let: 

Ci = {E G C{V k * Z n ~ k ), \E\ = i} 

And let: 

\d\ 



(\v\-iy(i) 

In particular, cq = 1. Let us show that for i > 0, Cj < 1, or in other words, that 
C(V k * Z n ~ k ) does not contain all the sequences of weight i. Of course, c\ = since d q > 2. 
Let us consider the case i > 1. If C('P fc * JJ"^) contains the two sequences of weight i: 

E 1 =X i .I n - i ,&nd 

e 2 =z.x i - l .r- i 
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then since it is a subgroup of V n , it contains the sequence of weight 1: 

E x * E 2 X = Y.I n ~ l 

This contradicts the fact that d q > 2. Thus, E\ and Ei are not both in Cj, and Cj < 1. Let 
c = max{c^\ 1 < i < n}, so that c G]0, 1[ and for all i £ [0, n], Cj < c\ Now consider an 
input sequence E € V Nk x 2 Nn , where |£| L > and \C® N {E)\ = d. The output C m {E) 
is the concatenation of N sequences in C(V k * Z n ~ k ) such that the sum of their weights is 
equal to d. Thus: 

N N 

a ® N {d)< y, ni c *i^ e n^d^i-i)* 

(di,...,d N )i=l (d 1 ,...,d N )i=l 
J2di=d J2di=d 
0<di<n 0<d z <n 

The product of the terms c di is always equal to c d . The product of the remaining terms, 
summed over (di, ...,dAr), counts exactly the number of sequences of size ./Vn and weight d. 
This proves the lemma. 

□ 

6 Upper bounds on the distance of a turbo-encoder 
6.1 Formal definition of a turbo-encoder 

A turbo-encoder T/v of size N is a concatenation of three operations. It requires a {n ou t, k ou t} 
encoder C ou t, an interleaver II, and an [nj n , fcj n , mj„] encoder Cj n . An interleaver is the 
following operation: 

Definition 6.1. An interleaver II of size N is an automorphism of V N composed of two 
operations: 

• First, a permutation ir of the N positions of the letters of the sequence 

• Second, a sequence (iri, 7Tjv) of N automorphisms of V , applied at each one of the 
letters 

Thus an interleaver H transforms a sequence E = E\ En into: 

^i(E n (i)) ttn(E^n)) 

The set of interleavers of size N is noted Pjy. 
Definition 6.2. Consider: 

• Cout an {n ou t,k ou t} encoder and Ci n an [n, k,m\ encoder 

• N = N out € N* and N in € N* such that AT out n out = N in k in + m in 

• II an interleaver of size N out n out . 




16 



The turbo- encoder Tjy based on C ou t, II, and C{ n , is an \Ni n ni n + rrii n , N out k out } encoder 
which does the following transformation. Consider a sequence E G ■p N outk ou t x f>N in (n in -k in ) 
written in the form: 

E = (Li, L Nout ,Si, S Nout ,S[, S^-.J 
TTien T^(E) is obtained by these three steps: 

• First, apply CfJ^ at the first 2N errors of E: 

E> = Cf^(Li, L N , Si, Sn) 

• Then, apply U at E' : 

E'perm = K(E') 

Since N out n out = Ni n ki n + mj n , E' perm can be written in the form: 

E'perm = (M' , L[, L' N . n ) 

where M' is of size mi n and all the other errors Li are of size ki n . Note that the first 
mi n letters of E' perm are specialized into memory letters for the next step. 

• Finally, insert (S[, ...,S' N ) into E' perm by putting each S^ after L\, and apply Ci n ^ in : 

Tn(E) = C inN . n (M' , L[, S[, L' N . n , S' N .J 

Note that even the memory part of this last output counts as part of the physical output 
with respect to the global protocol. 

A random turbo- encoder based on C ou t and C,- m is a turbo-encoder based on C ou t, an interleaver 
II chosen randomly with a uniform distribution over P^ nout , and Ci n . N is called the length 
of the turbo- encoder, and an integer N is said to be eligible for C ou t and Ci n if there exists 
an integer Ni n such that Nn out = Ni n ki n + mi n . 

If we go back to describing the real protocol lying behind a turbo-encoder, a state of 
information of size N ou tk ou t is encoded into a state of size iVj n nj n + nii n , by first encoding it 
into a state of size N out n ou t = Ni n ki n + mj n , then interleaving the positions of the obtained 
state, then encoding it again into a state of size Ni n ni n + rrii n . At each of the first and the 
last steps of the encoding, ancillary positions are added before the encoding is done. This is 
what corresponds, in the formal protocol, to the insertion of the stabilizer errors Si and S[. 

From now on, for a given turbo-encoder Fjy and a given input sequence E of the turbo- 
encoder, we will systematically write E' and E' perm to refer to the intermediate sequences 
obtained after applying CfJ^ and after applying II. 

6.2 Sketch of the counting argument 

We will sew the results obtained on the weight distributions for the inner encoder aN in (w, d), 
a N in {w, < d), and the outer encoder a® N (d), a® N (< d), to obtain the desired upper bounds 
on the distance of the turbo-encoder. For this purpose, the tools will be the two following 
lemmas. Let C ou t be a {n out , k ou t} encoder and Cj n be an [nj n , fcj n , mj n ] encoder. Let T/v be 
a random turbo-encoder based on C ou t and Cj n where iV is an eligible integer for C ou t and 
Ci n . Consider the probabilities related to the following events: 
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• PN(d) and pn(< d): there exists a harmful input sequence E such that, respectively, 
\T N (E)\ = d and \T N (E)\ < d. Note that p N {< d) is the probability that d c {T N ) < d. 

• pjy(w,d) and pn(w,< d): there exists a harmful input sequence E such that \E'\ = w 
and, respectively, \T N (E)\ = d and \T N (E)\ < d. 

Since the interleaver II is chosen at random, these probabilities are only functions of 
C ou t, Ci n and N. These probabilities are defined for all integers w and d, and j»at(< d) and 
Pn(w, < d) are also defined when d is real. 

Lemma 6.3. 

a® N H a Nin (w,d) 



V(w,d) G N x N, p N {w,d) < 



V(tu, d) € N x M, pjv(u;, < d) < 



d^i-irrr*) 

a 07V H a Nm (w,<d) 



(\v\-ir( Nn r) 



Proof. Let us prove the first inequality, the second one can be obtained with a similar reason- 
ing. p^(w,d) is the probability that there exists an input sequence E for the turbo-encoder 
Xjv such that \E\l > 0, \E'\ = w and \Tn(E)\ = d. In order to obtain this, a necessary con- 
dition is that the first 2N errors of E constitute one of the a® N (w) harmful input sequences 
of C®% such that the output (by C®^) has weight w. Take such a sequence of 2N errors. 
E' is now uniquely defined. Since 

^^P&T'TTb tfa.6 image of E' under the action of a random 
interleaver II, it is uniformly distributed over the set of sequences of size Nn ou t and weight 
w. Having \Tn(E)\ = d implies that E' perm is one of the aN in (w,d) sequences of weight w 
which are part of an undetected input sequence of Ci n ^ in such that the output by Ci n ^ in has 
weight d. This has probability: 

ajv in (u>,d) 



(pi-rrr) 

The inequality of the lemma is proved using the union bound, by summing this probability 
over all the a® N (w) possible values of the first 2N errors. □ 

The second lemma is straightforward by using the union bound: 

Lemma 6.4. For any real number D, and for all x G]0, 

[D\ [D\ ixN]-l [D] Nn out 

Pn(< D) <J2pn(w,< d) + ^2 PN(w,d) + ^2 ^2 p N (w,d) 

w=0 d=0w=[D\+l d=0w=[xN\ 

We call these three terms first, second and third partial sum. 
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Proof. 

Nn out 

Pn(< D)< Pn(w,<D) 

w=0 

ID\ Nn out 

<^2pn(w,<D)+ Pn(w,<D) 

w=0 w=[D\+l 
ID\ ID} Nn out 

<J2pn{w,<D) + J2 ^ Pn{w,oI) 

w=0 d=Ow=[D]+l 

ID\ [D] [xN\-l [D\ Nnout 

<^2pn{w,<D) + J2 Pn{w,oI) + J2 Y PN{w,d) 

w=0 d=Ow=[D\+l d=Ow=[xN\ 

□ 



6.3 The polynomial bounds 

We will start by proving case 1 of Theorem 11.11 then Theorem 11.21 since these results are close 
to each other, then we will prove case 2 of Theorem 11.11 The first two results are obtained 
using the three partial sums when D = N a . With the three following lemmas proved in 
Appendix B.l, we show successively that under the corresponding conditions, each of these 
three partial sums tends to as N — > oo. We write d c and d q respectively for the distance 
and the degenerate distance of C ou t- 

Lemma 6.5. First partial sum, poly case If d q >2 and C{ n is recursive, for all a < dq d 2 : 

\_N a \ 

lim V p N {w,< N a ) = 

io=0 

Lemma 6.6. Second partial sum, poly case If d q > 2 and Ci n is totally recursive, then 
for all a < dq d 2 , there exists x > such that : 

[N a \ [xN\-l 
\ im PN(w,d) = 



iV->-oo 

d=0 w=[N a \+l 



Lemma 6.7. Third partial sum, poly case If \V\ > 2, d q > 2 and Ci n is totally recursive, 
then for all x > 0: 

[N a \ Nn out 

lim V V p N (w,d) = 



jV->oo 

d=0 w=\xN\ 



The proof of Theorem 11.11 case 1 is now the following. 

Proof. The hypothesis of the theorem are that \V\ > 2, d q > 2, C{ n is recursive and C,- m is 
totally recursive. The conditions are met so that the results from the lemmas above apply. 
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For all a < 



dq — 2 
d q 



let x such that the second partial sum goes to 0. Let us write: 



YN a \ LA^J [x-M-1 L^J Nn out 

PN{<N a )<Y,PN{w,<d)+Y, Yl PN(w,d)+Yl J2 PN(w,d) 




Since the three partial sums go to 0, this proves that pn(< N a ) goes to as N — > oo. □ 

The proof of Theorem 11.21 is the following: 

Proof. The hypothesis of the theorem are that d q > 2, and Ci n is recursive and systematic. 
Being systematic implies that if w > d, pN{w,d) = 0. Thus the second and the third partial 
sums are equal to 0. The conditions of Lemma 16.51 are met and thus again, p_/v(< N a ) goes 
to as N — » oo. □ 

6.4 The sublogarithmic bound 

Now let us focus on Theorem 1 1 . 1 1 case 2. For this purpose we consider the three partial sums 
when D = log log N. The following proposition justifies the sublogarithmic expression of the 
bound. This property is needed in order to upper bound the first partial sum. 

Proposition 6.8. : Let us write, for N > 1, llog N = log iV/log(log N). For all t > 0, if N 

is big enoughf precisely, if N > e e ): 



Actually one can prove that the solution of the equation x x = N l is equivalent to 
t log N/ log (log N), and thus the bound proposed is the best asymptotic distance which can 
be proved with the arguments presented in this paper. The three following lemmas proved in 
Appendix B.2 show under the corresponding conditions that each of the three partial bounds, 
when D = log log N, tend to as iV ^ oo. 

Lemma 6.9. First partial sum, sub-log case If d c > d q = 2 and Ci n is recursive, for all 
a < d c — 2: 



(t llog N) 



t llog N 



< N '' 




<tlogN 



□ 



[a llog N\ 



lim 

N->oo 




Pn{w, < a llog N) = 



w=0 
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Lemma 6.10. Second partial sum, sub-log case If d c > d q = 2 and Ci n is totally 
recursive, for all a < d c — 2, there exists x such that: 

[a llog N\ [xN\-l 

lim V V p N (w,d)=0 

TV— >oo * — ' * — ' 

d=0 v}=\a llog TVJ + 1 

Lemma 6.11. Third partial sum, sub-log case // \V\ > 2, d q > 2 and Ci n is totally 
recursive, then for all x > 0: 

[a llog TV J Nn out 

lim V V p N (w,d)=0 

TV-J-oo * — » z — ' 
d=0 w=[xN] 

The proof of Theorem 11.11 case 2 is now the following. 

Proof. The hypothesis of the theorem are that \V\ > 2, d c > d q = 2, Cj n is recursive and Ci n 
is totally recursive. The conditions are met so that the results from the lemmas above apply. 
For all a < d c — 2, let x such that the second partial sum goes to 0. Let us write: 

La llog TV J |a llog TV J [xN]-l [a llog TV] Nn out 

p N (a llog N) < ^2 Pn(w, < a llog N) + p N (w,d) + p/v( 

w=0 d=0 io= [a llog TVJ+1 d=0 i/j=|xTVJ 

Since the three partial sums go to 0, this proves that pn(& Hog N) goes to as N — > oo. □ 



Appendix A: Bounds for the inner convolutional encoder 

We give here the proof of Theorem 14.61 Consider a sequence (M, L\, L/y) of weight w, 
which is part of E = (M, Li, Si, L^, Sjy) such that |Cjv(i£)| < d, and let (Pi, P/v, M') = 
Cn(E). To begin with, suppose that the information part is of given weight \(L±, Ljv)| = 
wl, and suppose that the trace of E is made of c detours. We remind the following notations. 
1 < Pi < ■■■ < Pw L < kN are the positions of the wl non identity letters of (L\, ...,L/y), and 
for each i E [1;iul], iVj = \pi/k] is the index such that the ith non identity letter is part of 
the error L^. r The ith truncature of E is the sequence: 

E\i = (M, Li, Si, LjVi-D S'iVi-n L N ., SjvJ 

where (L\, L^ i _ 1 , L' N _) is obtained by replacing all the letters in (L%, LjvJ after the 
position pi by an I. Also, po = 0, iVo = and E\n = M. Moreover: 

Mi = fJ, Ni (E\i) 

and for i = 0, Mo = M. The trace of E is the sequence (bo,..., b WL ) such that for i € [l; wj, 
Mj G Mb-. For i S [1; c], i>j is the starting point of the ith detour, and v c+ \ = wl + 1. 

We will start by confining the space where lie the positions of the non identity letters of 
(L\, ...,Ljv), by cutting them into packets corresponding to each detour. For each i £ [l;c], 
let 5p( % ' = — Pvi- The sum of the sizes of these intervals is upper bounded by: 
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Lemma 6.12. If wl > 1- 



5p^> < min(kN, r]k(wL + d)) 



i=i 



w/iere <5p (i) =p„ i+1 _i 

The main idea is to prove that the difference of weight between the outputs at the steps N Vi 
and N Vi+1 of the convolutional operation is proportional to 6p® . The main argument is that 
for j from Vi to Uj+i — 1, the memory errors Mj all belong to Mi, which by the recursiveness 
of C yields an output weight proportional to the input size during that time. We first apply 
this idea between the steps Nj and Nj + i — 1 of the encoding for a given j G {vq; wj: 

Sublemma 6.13. Let j G [uo;^l]- 

(\P Nj+ i,...,P Nj+1 _i)\> Nj+1 -^ 1 
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Proof. If iVj+i = iVj- or A/j + 1, this is true since the sequence (Pjv.+i, P/v 3+1 -i) is empty, 
and its weight is non negative. 

Now suppose that AT ? - +1 > A/j + 1, and let us first consider the case j > 0. Notice that 
L' N , = Ljsfj, because the non identity information letter in position Pj+i belongs to the error 
I j N--\_\ , which comes strictly after the error Ljy-- We can write the following concatenation: 



E \j+1 ~ E \j ■ i 1 > S Nj Sn j+1 -1, L N j+1 . S N J+1 ) 

The physical output (Pi, Piy, + i-i) is produced by applying the encoder at the first 2Nj + \ — 1 
errors of -E\j+i- By concatenation, this output can be written: 

(Pi, ...,PjVj+i-l) = TtN j+1 -l{E\j-{I, SNj+l, SN j+1 -l)) 

= 7T jv,- ( E\ j ) . 7T Nj + 1 _ iv,- - 1 ( Mj , / , Sn 3 + 1 , • • • , I , + x - 1 ) 

= (Pi, ■■;PN j )-^N j+1 -N j -l(Mj,I, SNj+1, SjVj_|_i— l) 

This shows that: 

(PVj+l, PjVj+i-l) = K Nj+1 _ Nj _i(Mj , I , SNj+1, ...,I,SN j+1 -l) 

In the case where j = 0, this equality also holds and has the form: 

(Pl,— ,Pzvi-i) = 7i"JVi-i(M ,7, Si,..., I, SW1-1) 
Since Mj G Mi and C is recursive: 

KP^+i, ...,p?v j+1 -i)| > L J 

And this implies the desired inequality. □ 
Now by simply summing over j we obtain the proof of the lemma. 
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Proof. Let i £ [1; c]. The sum of the inequality in the previous lemma over all the values of 
j € \vi]Vi + i — 2] gives the result: 

\(P Nvi , > N "+i-*- N " - ( Vi+1 - Vi -1) 

Using AT„. < p„./A; + 1 and iV„ i+1 _i > p Vi+1 ^/k we get: 



|/ D D \ I \ Pv i+ 1-1 Pvi 1 , -| \ 

KP^,...,^^^)! > ^ - - Vi - 1) 

By summing again this inequality over i £ [1; c] we get that: 
This implies the first bound: 



i=i 



e-1 



^25p (i) < 7]k(w L +d) 

The second bound is obvious: 

c c 

X] 6 P {l) = J2(P^+i-l ~Pvi)< Pw L - Pv c + J2(Pv i+1 - Pvi) < kN 
i=l i=l i=l 

□ 

This upper bound is useful in that it says that all the non identity letters of (L±, Ln) 
starting from the letter in position p Wl (ie, from the first non identity letter since w\ = or 
1) are confined in a bounded region of space. Now we can prove the following bound: 

Lemma 6.14. Suppose wl > 1, and suppose that c is fixed. The number of possible values 
for the sequence {L\, ...,Ln) is upper bounded by: 



(\r\-r 



WL 



wl\( kN\ ( min (kN, r]k(w L + d)) + 1 
c — 1/ V c / V wl - c + 1 



Proof. There are two possible values for the starting point w\ of the first detour, either or 
1. In both cases, the number of possible starting points Wi, 2 < i < c of the c — 1 remain- 
ing detours is upper bounded by ( ( ^_ L L ) , since the c — 1 remaining values are in the interval 
[1, v c j. For each choice of these starting points, the number of possible values for the positions 

(Pvi)i<i<c, is at most ( k ^). 

Now, consider that the sequences (fj)i<i< c and (p Vi )i<i<c are fixed. The number of 
positions of the non identity letters in (L±, Ljv) which are still unfixed is equal to wl — c 
if v\ = and wl — c + 1 ift>i = 1. This sequence of remaining positions can be written: 

(Pj) Vi<j<v i+1 , l<i<c 

where as defined previously, v c+ i — 1 = wl- In order to upper bound the number of such 
possible remaining sequences, we will rely on the fact that ^i=i > which we know is 
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upper bounded, is intuitively the space in which the remaining positions are confined. Since 
this sequence is strictly growing, it is equivalent to upper bound the number of possible 
corresponding sets. Such a corresponding set can be written as: 

1=1 

where: 

2» = {pj,Vi < j < v i+1 } . 

In the following, we exhibit a reversible transformation of this set into a set of equal number 
of elements and included in the interval [l,X^i=i ^P^l- Define si = and for i £ [2,c + 1], 
let Si = Y2\~=i dp^ 1 '^ ■ Consider the transformation which transforms the set of remaining 
positions into : 

US 

i=i 

where: 

% = {Pj = Pj ~ Pvi + Si, Vi < j < v i+ i} . 

This function relies on the knowledge of Vi and p Vi . Let us show that this transformation is 
injective by showing explicitely how to recover the antecedent of a given image set. 

First, let us show that in the image set, any two elements pj and pj> respect the order of 
their indexes, ie they verify pj < pj/ if and only if j < j'. 

For each i 6 [1, c], and for each couple (pj,pj>) £ if where j < j': 

pj > 1 + Si because pj — p Vi > 1 

Pj' < Si + i because pji — p Vi < Sp^ 

Pj < Pj' because pj < pj> 

If we compare two elements in a same set 2j, the third inequality proves that they respect 
the order of their indexes, whereas if pj £ 2j and pj/ G 2j/ with i < i', we can use the first two 
inequalities to get: 

Pj < S i+ \ < 1 + Si> < 1 + pj' 

This implies that, by looking at the position of an element respectively to the others in the 
image set, we know necessarily the value of its corresponding index j. Next, the c + 1 values 
of Si can be recovered recursively as follows. We start with s\ = 0. Then, for each i G [l,c], 
either Xj is empty, in which case Wj+i = Vi + 1 and Sj + i = Sj, or it is not, in which case Sj+i 
is equal to the last element of %: p Vl+1 -i = Sp^ + Si = Sj+i. Now that all the values of pj 
are known, and since the values of p Vi are also known, all the values of pj can be recovered. 
Finally, the fact that all the values pj are confined in the set [l,X)i=i $P^} comes from the 
first two inequalities. 

This proves respectively that, when v\ = and when v\ = 1, the number of possible 
sequences of remaining positions (Pj)vi<j<v i+1 , i<i<c is upper bounded by the number of 
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subsets of [1, min (kN, rjk(wL + d))J of respective number of elements wl — c and wl — c + 1. 
This gives two binomials, the sum of which is: 

min (kN, nk(wL + d)) + 1 
wl - c+ 1 

Finally, the proof is completed by noticing that each of the wl non identity letters takes at 
most \V\ — 1 values. □ 

Now let us release the constraint on the number of detours c: 

Lemma 6.15. Suppose that wl > 0. The number of possible values for the sequence (L\, Ljy) 
is upper bounded by: 



2 wl (\v\ - r 



kN + 1 \ /min (kN, nk(w L + d)) + 1 

m + v v m 



Proof. The sequence of N identity errors is the only one such that wl = 0. Thus if wl = 0, 
the bound is true. Now we suppose that wl > 1. For a purpose of lisibility, let B = 
min (kN, nk(uiL + d)) + 1, and let c max = |_^J + 1 be the maximal value for the number c of 
detours. By the previous lemma, he number of possible values for the sequence (L\, Ljv) 
is upper bounded by: 



( - m a -v 



(w\ - ir- Yl 



c=l 



w L \ fkN\ ( B 

max 



1 I l<c<c max \ C J \WL — C + 1 



Replacing kN by kN + 1 in the previous expression keeps the inequality true and makes the 
analysis easier. Let us maximize over c, and under the constraint c < c max , the expression: 

(kN + l\( B 

For all c < c max — 1, the ratio r(c) between two successive values is: 

/(c + 1) kN + 1 - c w L - c+ 1 



r(c) 



/(c) B-(w L -c) c + 1 



When c < c max — 1, c < wl/2, and thus c < wl — c. Besides, B < kN + 1. Thereby, r(c) > 1 
for all c < Cmax — 1) which proves that the maximum is reached at c = c max . When c takes 
this value, wl — c + 1 is equal to |"^]; this comes from the fact that [^J + T^l = W L- 
Thus: 

/ kN\ ( B \ _ ( kN + l\fmin(kN,nk(w L + d)) + 

iSLA c ) \w L -c+i)~ vm + 1) v m 



Moreover: 



Cmax / \ 



c=l 

and this proves the lemma. □ 
Finally comes the proof of Theorem 
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Proof. The number of memory errors M of given weight wm is: 

(\V\ - l) WM 



m 

WM 



Thus by the previous lemma, the number of possible sequences (M,L\, ...,Ljv) verifying the 
conditions of the theorem and such that \M\ = wm is upper bounded by: 



OPl-lW m )g(w L ) 
\wmJ 

where wl = w — wm and where the function g is given by: 

, 1 ,™ { kN+l\ (r,k(w + d) + 1 

Again, we search for the maximum of g for all values of wl under the constraint wl < w < kN. 
If wl < w — 1 then these two inequalities: 

L^J + i<^ + i) 

and: 

< ^Hw + d) + 1) 

show that g(u>L + 1) > g(wi). Then, g reaches its maximum at wl = w. The proof is 
completed by the majoration: 

m 



E H 2 



rn 



w M =0 



Appendix B: partial sums 

B.l: Polynomial case 
Proof of Lemma l6.5t 

Proof. First partial sum, poly case Let d = N a . We use the following property: 
with the following bound derived from the bound (IE), valid because d q > 2: 

w 

a m (w) < 0{l) w (^j d " if w > d c 

= ifw; < d c 

and the bound (II), valid because Cj n is recursive: 

w 

,Ng{w + d)2 



□ 



a Nin (w,<d)<0{lf 
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Using the fact that w < d and iVj n = 0(N), this bound gives : 

N 2d~ 

a Nm (w,<d)<0(iy 



We lower bound the denominator using the following lower bound on binomials: 

V(«,u) € N 2 /v < u, 



which, together with the facts that \V\ > 2 and n ou t > 1, implies : 



(\V\-1 

As a consequence we obtain, when w > d c : 



An ou t 



> 



. N\ *i Nzd? f N 



Til / ID" 

it; it? 



<0(l) u, A^ d 9 NidiN 



< 



0{l)N d <i 2 d* 



and when iu < d c , pn(w, < d) = 0. Let us simply replace d by its value A°: 
The exponent verifies: 



la 1 1 / 2 — c/, 



d q 2 2 2 



+ a < 



And since pjsr(w, < A a ) = when to < d c , this implies: 

<iV a )< ^ (o(l)A 



1 . a 1 \ tU 



u>=0 ui=d c 

< 0{l)N y 

This proves that the first partial sum tends to as N — > oo. 

Proof of Lemma 16.61 

Proof. Second partial sum, poly case We use: 

a® N (w) a Nm (w,d) 
PN{w,d) < — r- 

(\r\-i) w ( N T) 

with the same bound as before derived from (IE), valid because d q > 2: 



a m (w) < 0(1) 1 



= 



w 



if w > d c 
iiw < d r 
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and with the bound (21), valid because Cj n is totally recursive: 



a Nin (w,d)<0(iy 



d d 



Since iVj n = 0(N) and w > d, this last bound can be written in the following form: 



, T d d 



a Nin (w,d)<0(l) w —^ 

Let us also lower bound the denominator as previously by: 

'N^ w 



w 



As a consequence: 



p N (w,d) <0(1) 



. _u>_ d d 
N\ dq N2W2 f N 



< 



w 



w 



w 



d d 



w d d 



d d 



Let us upper bound O(l) by a constant a > 0, and take the logarithm of the last inequality: 

log(pjv(w, d)) < f(w, d) = w (log a + (1 - l/d q )(\ogw - log AT)) + - (log iV + \ogw - 21og<f) 

Let us show that there exists x sufficiently small such that, for iV sufficiently large, the 
maximum of f(w,d) in the domain (w, d) £ [[iV a J, [xN\ — 1] x [0, L^ a J] is reached when 
w = d = [N a \. First, consider the derivative of / with respect to w: 



logo + (1 - l/dg)(logio - log N + 1) + 
Since ij- <\ and w < xN, this is upper bounded by: 



2w 



loga + (l-l/d,)(logx + l) + - 

For a sufficiently small x, this upper bound is negative. Indeed, when x — > 0, the above 
expression is dominated by the term: 

(l-l/dg)IOgX 

which tends to — oo. This shows that, provided x is small enough, f(w,d) is maximum when 
w = [A^J • Now, suppose that w = [N a \ , and consider the derivative of / with respect to d: 

^ (log N + log [^"J - 2 log d - 2) 

This is a decreasing function of d. Thus the minimum of this derivative is reached at d = \_N a \ 
and is equal to: 



1 



(log AT -log [AH -2) 
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When N — > oo this minimum is equivalent to: 

1 — a 



logiV 
2 

Since a < 1, for a sufficiently large N this minimum is positive. Consequently f(w,d) is 
maximum when w = d = [N a \ . The maximum of / in the domain is equal to: 



[N a \ 

T 

:LAH(loga + (1/2 - l/^)(logLAH - log AT)) 



f([N a \, MV°J) =VN a \(\oga + (1 - l/d q )(\og[N<*\ - log AT)) + ^(logAT - log^J) 



When A^ — > oo this maximum is equivalent to: 

/(LAH, LiV a J) ~ -(1/2 - l/d q ){l - a)N a logiV 

Now in the sum: 

[N a i [xN\-i 

£ PN(w,d) 

d=0 w =[N a ]+l 

the number of terms is upper bounded by xN 2 . Thus the logarithm of the sum is upper 
bounded by: 

log x + 2 log N + /( LiV Q J , LAH ) ~ "(1/2 - - a)iV Q log A" 

This equivalent tends to — oo, which proves that the second partial sum tends to if we chose 
a sufficiently small x. □ 



Proof of Lemma [67 
Proof. Third partial sum, poly case Again we use: 



a® N (w) a Nin (w,d) 

PN(W,d) < — 7^ 

{\v\-i) w ( Nn r) 



According to the bound (2E), valid because \V\ > 2 and d q > 2, there exists a constant 
c €]0, 1[ such that: 

a® N (w) <c w (\V\-l) w ( Nnouty 



w 

Let us also use the bound (21), valid because Ci n is totally recursive: 

Since Aj n = O(N), d < N and it; < Nn ou t, this bound gives: 

a Nm (w,d)<0(l) d N d 

With these two bounds we obtain: 

p N {w,d) < c w O{l) d N d 
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Thus: 

log(p' N (w,d)) < wlogc + d(0 (I) + log N) 

< [xN\ log c + N a {0(l) + log N) 

where the last line is obtained using logc < 0, w > [xN\ and d < N a . Since the third partial 
sum contains less than n 0U tN 2 terms, its logarithm is upper bounded by: 

log n out + 2 log N + log(p' N (w, d)) < log n out + 2 log N + [xN\ log c + N a (0(l) + log N) 

Since a < 1, this upper bound is equivalent to: 

:ciV log c 

and this proves that the sum tends to when N — > oo. □ 

B.2: Sub-logarithmic case 
Proof of Lemma 16.91 

Proof. First partial sum, sub-log case Let d = cdlog N. We use: 

a® N (w) a Nm (w,<d) 
Pn(w, < d) < — ^Vkr H 1 

(i^i-irr n r) 

Let us use this result from the bound (IE), valid because d q > 2: 

w— d n i i 



a^H < O(l) 1 

' to 

and the bound (II), valid because Cj n is recursive: 



a Nm {w,<d)<0{iy 



Since w < d and Ni n = 0{N) this bound gives: 

N *d~ 

a Nm (w,<d)<0(iy 



w" 

Let us also lower bound the denominator: 

n v \-ir( Nn ° ut ) >(* 

\ w J \w 
By combining these three bounds we get: 

PN (w,<d)<0(l) w (-) — - 

w — d c | ^ ^ w t w 



<0(l) w N— +1 N2d2N~ 
<[0(l)d]fiV^ +1 
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The last bound is an increasing function of w. Applied at w = allog N, it is thus a bound 
for each term of the first partial sum: 

[a llog N\ 

Ea llog N d r i -1 

Pn(w, < a llog N) < (allog N + l)[0(l)a Hog iV] — 5 — AT--+ 1 

uj=0 



a llog n / a lose iV \ Q(i) 

O(l) - r— = exp O(l) " AT . = iV^ia^) 
K ' V ; 21og(logiV); 



Now, notice that: 



And using Proposition! 

a llog iV ol 

[a llog N] 2 < 

Mutliplying these terms we get: 

10 U ° g Nl O(l) 

^ Pat(w, < a llog iV) < (allog N + l)ATiog(iogJv) i\r £L r k+1 

w=0 

Since 01 "Tj c + 1 < the first partial sum tends to when iV — » oo. □ 

Proof of Lemma I6.1UI 
Proof. Second partial sum, sub-log case We use: 

(pi - irCr*) 

Let us apply this result from the bound (IE), valid because d q > 2: 

a m {w) < 0(l) w ( — 
\ w 

and the bound (21), valid because Cj n is totally recursive: 

a N m { w i d ) < °W J d 

Using the facts that Ni n = 0(N) and w > d, this bound gives: 

w d 
N 2 W 2 

We also lower bound the denominator: 

\ w J \w 

Combining these bounds gives: 

w—dp . -i w d , 

N\—^ +1 N2 W 2 ( N 



p N (w,d) <o(iy 



<AT-f+ 1 



w l d d \ w 



0(1) 



i 



w 



, T d d 
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Let us upper bound O(l) by a constant a > 0; then consider the logarithm f(w,d) of the 
right hand expression apart from the first factor: 



f(w,d) = log 



i 



N\~ 2 
w 



w d d 

d d 



= w(loga + 1/2 (log w — log N)) + ^(log A" + logw — 21og d) 

Let us show that there exists x sufficiently small such that, for A^ sufficiently large, the 
maximum of f(w,d) in the domain (w,d) £ {[a llog N\; \_xN\ — 1] x [0; [a llog A r J] is 
reached when w = d = \a llog N\ . The derivative of / with respect to w is equal to: 

loga + l/2(logu; - logN + 1) + — < log a + l/2(log x + 1) + £ 

where the inequality comes from the facts that w < a; AT and d < w. For a sufficiently small 
x, the right hand side (and thus the partial derivative) is negative. This shows that the 
maximum of f(w,d) is reached if w = \a llog N\. Moreover, there exists r g]0; 1[ such that 
the derivative of / with respect to w is upper bounded by logr. By integrating this inequality 
from \a llog N\ to w we get the following inequality: 

f(w, d) <f([a llog N\,d) + log r (w - [a llog A^J ) (1) 

Now suppose that w = [a llog N\ , and consider the derivative of / with respect to d: 

- (log N + log [a llog N\ —2 log d — 2) 

This derivative is a decreasing function of d; its minimum is reached when d = \a llog N\ 
where it is equal to: 

i (log A" - log [a llog N\ -2) 

which is positive for A r large enough. Thus f(w,d) is maximum when w = d = [a llog N\. 
The maximum of / in the domain is then equal to: 

I a llog N I 

La llog N\(loga + 1/2 (log La llog N\ - log AT)) + ± ^ — ± (log AT - log La llog JVJ) 

= La llog N\ log a = 0(1) llog A" 
Thus by inequality [H we get for all (w,d) in the domain: 

f(w, d) < O(l) llog AT + log r (to - La llog N\ ) 
This proves in turn that : 

p N (w,d) < N'^ +1 N^VoeN) r w~[a Hog N\ 
Thus for each d £ [0; La llog N\J, since r < 1: 

NVJ-l oo Q 

^ p N (w,d)< ^ N-f + 1 N^do g N )r w~la llog JVj 

iw=|_a llog ATJ+1 iu=[_a llog N\ 

^©(l)^"" 4 " 1 ^ 1 ^^^) 
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Consequently, by summing over d, the second partial sum is upper bounded by: 

O(l) (Hog N + l)N-^ +1 N^°°z N ) 

which tends to when N — > oo since d c > 2. □ 

Proof of Lemma [6.111 Third partial sum, sub-log case 

Proof. The proof is exactly the same as the third partial sum in the polynomial case, by 
changing every occurrence of N a by a Hog N. □ 
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